Virtual environments have made significant advances in recent years. With those advances come increasing demands from users to increase interactions with the virtual environment. Real-time control of pre-determined models using web-cam based head tracking is currently available in some video chat applications and allows a user to control a virtual avatar head by tracking the position of the user's head and using the user's voice to control the mouth of the avatar. In this case though, the avatar is pre-determined, tracking is limited to the head, and a simple one-to-one mapping between the user's head orientation and the avatar's head orientation is used. There is currently not a model that allows a user to fully control an object that has been received from the real-world and digitized into a virtual object. In other words, an object that is not pre-determined. Further, utilizing skeletal tracking information of a user to control the entire virtual object (e.g., limbs, the head, etc.) using the user's body is not available.